BUG: resolve divide by 0 error when uploading empty dataframe #252
Conversation
bwanglzu
changed the title
|
Thanks a lot @bwanglzu Would you be able to add a test? It should be very short |
|
@max-sixty definitely. |
tests/unit/test_gbq.py
Outdated
| @@ -246,6 +246,28 @@ def test_to_gbq_doesnt_run_query( | |||
| mock_bigquery_client.query.assert_not_called() | |||
|
|
|||
|
|
|||
| def test_to_gbq_uploading_empty_dataframe( | |||
There was a problem hiding this comment.
Very reasonable to copy code also testing writes, but this is doing something slightly more involved re pandas versioning
Could you copy the code here: https://github.com/pydata/pandas-gbq/blob/272aa7bebef5dc99869e7c44adf8011258c8d7c9/tests/system/test_gbq.py#L879 and put the test there?
There was a problem hiding this comment.
Hi @max-sixty thanks for your feedback. Two questions:
- Shall I create a test method in
pandas-gbq/tests/system/test_gbq.pynamedtest_upload_empty_data, or just create an emptypd.DataFramewithintest_upload_data? Which one is better? - Do I need to delete
test_to_gbq_uploading_empty_dataframeinpandas-gbq/tests/unit/test_gbq.py?
Thanks!
There was a problem hiding this comment.
Shall I create a test method in
pandas-gbq/tests/system/test_gbq.pynamedtest_upload_empty_data
This would be perfect!
That's the only test that's needed (no need to keep the existing test_to_gbq_uploading_empty_dataframe)
|
Great! |
|
Hi @max-sixty thanks for your guidance! If needed I'll add the changelog note. Pandas-gbq really helped me a lot, thanks for your great work! Best |
|
hi @max-sixty I did find out |
|
Hi @bwanglzu - under 0.10.0 (0.9.0 was already released), under a new section "Bug fixes". Thanks! |
|
@bwanglzu I think you need to merge master into your branch - the current changes to |
|
@max-sixty Oh sorry I didn't see your message.. I directly resolved the conflict on github UI |
docs/source/changelog.rst
Outdated
| @@ -1,6 +1,14 @@ | |||
| Changelog | |||
| ========= | |||
|
|
|||
| .. _changelog-0.11.0: | |||
|
|
|||
| 0.11.0 / 2019-02-25 | |||
There was a problem hiding this comment.
This should still be in 0.10.0!
|
Thanks a lot @bwanglzu ! Happy to have you as a contributor |
|
@max-sixty Thanks for your help! |
| def test_upload_empty_data(self, project_id): | ||
| test_id = "data_with_0_rows" | ||
| test_size = 0 | ||
| df = DataFrame() |
There was a problem hiding this comment.
Looks like we might have an additional problem when the DataFrame contains no columns.
In the conda build (https://circleci.com/gh/tswast/pandas-gbq/276) I'm getting:
E google.api_core.exceptions.BadRequest: 400 POST https://www.googleapis.com/upload/bigquery/v2/projects/pandas-gbq-tests/jobs?uploadType=resumable: Empty schema specified for the load job. Please specify a schema that describes the data being loaded.
Since we still create a table in pandas-gbq before running the load job, we can probably avoid doing the load job altogether when a DataFrame contains no rows.
For issue #237 .
A quick fix to resolve
ZeroDivisionErrorwhile uploading an empty dataframe.